Data processing apparatus and data processing method

ABSTRACT

A data processing apparatus ( 10 ) is a data processing apparatus ( 10 ) that determines a target region within a picture contained in a video content, on the basis of a trajectory input by a user in order to surround a moving object in the video content, the data processing apparatus comprising: an input unit ( 11 ) configured to receive input data showing the trajectory input by the user in the displayed video content; and a correction unit ( 12 ) configured to correct an input region, which is a region specified by the trajectory expressed by the input data, to thereby determine the target region.

TECHNICAL FIELD

The present invention relates to a data processing apparatus and dataprocessing method that determine a target region within a picturecontained in a video content on the basis of a trajectory input by auser in order to surround a moving object in the video content.

BACKGROUND ART

Recently, a user has dramatically increased opportunities to use a videocontent other than viewing of TV (television) broadcasting because ofdevelopment of the digital techniques and the Internet and popularity ofvideo posting sites. The user enters a keyword at a video posting sitesuch as YouTube (registered trademark) or a search site such as Google(registered trademark), and retrieves a desired content. Unfortunately,in this method, elements having no keyword cannot be retrieved.Accordingly, a method by which the video content can be retrieved moresimply and securely has been examined. For example, there has beenproposed a method in which other video content is retrieved directlyfrom one video content. As a conventional method for retrieving othervideo content from one video content, a method has been disclosed inwhich a video content is retrieved using part of a region of one stillpicture that forms the video content (for example, see Patent Literature1).

CITATION LIST Patent Literature

-   [PTL 1] Japanese Unexamined Patent Application Publication No.    2008-146491

SUMMARY OF INVENTION Technical Problem

In the conventional method, in the case where the user desires to payattention to one region within the video content for retrieval, the userneeds to designate an object by using an input means such as a mouse tosurround the object. The object is moving within the video content,however. For this reason, input of a trajectory in order to surround theobject is out of order, and as a result, the region specified by theinput trajectory is not always an appropriate region for retrieval.

Accordingly, in order to solve the problem above, an object of thepresent invention is to provide a data processing apparatus and dataprocessing method that can properly determine a target region within apicture contained in a video content on the basis of a trajectory inputby a user in order to surround a moving object in the video content.

Solution to Problem

In order to achieve the object above, a data processing apparatusaccording to one embodiment of the present invention is a dataprocessing apparatus that determines a target region within a picturecontained in a video content on the basis of a trajectory input by auser in order to surround a moving object in the video content, the dataprocessing apparatus including: an input unit configured to receiveinput data showing the trajectory input by the user in the video contentdisplayed; and a correction unit configured to correct an input region,which is a region specified by the trajectory expressed by the inputdata, to thereby determine the target region.

According to the configuration, the input region, which is a regionspecified by the trajectory expressed by the input data, can becorrected to thereby determine the target region. Accordingly, in thecase where the input region is not an appropriate region, the inputregion can be corrected, and the target region can be properlydetermined.

Preferably, the input data is data showing the trajectory input by theuser across several pictures contained in the video content; the dataprocessing apparatus further includes a movement calculation unitconfigured to calculate an amount of movement and moving direction of animage within the input region in the several pictures; and thecorrection unit is configured to correct the input region using thecalculated amount of movement and moving direction of the image withinthe input region, to thereby determine the target region.

According to the configuration, the input region can be corrected usingthe amount of movement and moving direction of the image within theinput region. Usually, the input region contains the object that theuser intends to surround. Accordingly, according to the configuration,the input region can be corrected according to the movement of theobject. Thereby, even if the movement of the object causes the input ofthe trajectory to be out of order, the input region can be correctedusing the movement, and the target region can be properly determined.Namely, the region containing the object can be properly determined asthe target region.

Preferably, the correction unit is configured to determine, as thetarget region, an intersection of a region after movement obtained bymoving the input region according to the amount of movement and movingdirection of the image within the input region and the input region. Inthis case, for example, the correction unit may be configured todetermine an intersection of the region after movement obtained bymoving the input region in the moving direction by the amount ofmovement and the input region as the target region in a final pictureamong the several pictures. Alternatively, for example, the correctionunit may be configured to determine an intersection of the region aftermovement obtained by moving the input region in a direction opposite tothe moving direction by the amount of movement and the input region asthe target region in an initial picture among the several pictures.

According to these configurations, the intersection of the region aftermovement obtained by moving input region according to the amount ofmovement and moving direction and the input region can be determined asthe target region. Accordingly, even if the input region becomes largerthan necessary because the user follows the movement of the object, theregion that contains the object and is smaller than the input region canbe determined as the target region. As a result, the target region canbe properly determined.

Preferably, the correction unit is configured to compress the inputregion according to the amount of movement and the moving direction, tothereby determine the target region. In this case, for example, thecorrection unit may be configured to compress the input region in themoving direction by the amount of movement, to thereby determine thetarget region in the final picture among the several pictures.Alternatively, for example, the correction unit may be configured tocompress the input region in the direction opposite to the movingdirection by the amount of movement, to thereby determine the targetregion in the initial picture among the several pictures.

According to these configurations, the input region can be compressedaccording to the amount of movement and moving direction to therebydetermine the target region. Accordingly, even if the input regionbecomes larger than necessary because the user follows the movement ofthe object, the region that contains the object and is smaller than theinput region can be determined as the target region. As a result, thetarget region can be properly determined.

Preferably, the movement calculation unit is configured to select atleast one pixel group in at least initial one picture among the severalpictures on the basis of the input trajectory, and calculate an amountof movement and moving direction of the selected pixel group as theamount of movement and moving direction of the image within the inputregion.

According to the configuration, in at least initial one picture amongthe several pictures, at least one pixel group can be selected on thebasis of the input trajectory. Accordingly, the pixel group can beselected on the basis of the input trajectory before the object moveslargely. If the amount of movement and moving direction of the imagewithin the input region is calculated using the pixel group thusselected, the amount of movement and moving direction thereof can becalculated according to the movement of the object. Accordingly, if theinput region is corrected using the amount of movement and movingdirection thus calculated, the target region can be properly determined.

Preferably, the movement calculation unit is configured to calculate theamount of movement and moving direction of the image within the inputregion concurrently with the input, and the correction unit isconfigured to correct the input region concurrently with the input.

According to the configuration, the input region can be correctedconcurrently with the input by the user.

Preferably, the input data is data showing the trajectory input by theuser across several pictures contained in the video content; and thecorrection unit is configured to correct the input region on the basisof a shape of the trajectory expressed by the input data after the inputacross the several pictures is completed, to thereby determine thetarget region.

According to the configuration, the input region can be corrected on thebasis of the shape of the trajectory, and calculation of the amount ofmovement and moving direction of the image within the input region isunnecessary. Accordingly, calculation load can be reduced.

Preferably, the correction unit is configured to approximate thetrajectory expressed by the input data to an ellipse, and determine aregion in the vicinity of a point of intersection as the target regionin a picture in which the point of intersection is input, the point ofintersection being a point of intersection farthest from the start pointof the trajectory among points of intersection between a long axis ofthe ellipse and the trajectory.

According to the configuration, the trajectory can be approximated to anellipse, and the region in the vicinity of the point of intersectionfarthest from the start point can be determined in the picture in whichthe point of intersection farthest from the start point of thetrajectory among the points of intersection between the long axis of theellipse and the trajectory is input. Accordingly, the region in thevicinity of a position in which the trajectory for surrounding theobject starts to be closed can be determined as the target region.Namely, because it is thought that the object exists in the position inwhich the trajectory starts to be closed, the target region containingthe object can be properly determined.

Preferably, the correction unit is configured to complement thetrajectory expressed by the input data, to thereby determine the targetregion.

According to the configuration, the trajectory input by the user can becomplemented to thereby determine the target region.

Accordingly, the target region can be properly determined before theobject moves largely.

Preferably, the correction unit is configured to approximate thetrajectory expressed by the input data to an ellipse, to therebycomplement the trajectory.

According to the configuration, the trajectory input by the user can beapproximated to an ellipse to thereby complement the trajectory. In thecase where the user inputs a trajectory in order to surround a movingobject, the shape of the trajectory is usually likely to be elliptical.Moreover, if the shape of the trajectory is elliptical, approximationcan be performed at the relatively small number of points. Accordingly,the trajectory input by the user can be approximated to an ellipse tothereby properly determine the target region.

Preferably, the correction unit is configured to determine an algorithmfor determining the target region on the basis of the input data, andcorrect the input region according to the determined algorithm, tothereby determine the target region.

According to the configuration, the algorithm for determining the targetregion can be properly determined on the basis of the input data, andthe target region can be more properly determined.

Moreover, the data processing apparatus may be configured as anintegrated circuit.

The present invention can be achieved as such a data processingapparatus, but also as a data processing method including operations ofthe characteristic components included in such a data processingapparatus as processings. Alternatively, the present invention can beachieved as a computer program causing a computer to execute each of thecharacteristic processings included in the data processing method. Sucha computer program can be distributed through a non-temporary computerreadable recording medium such as a CD-ROM (Compact Disc Read OnlyMemory) and a communicating medium such as the Internet.

Advantageous Effects of Invention

According to the present invention, on the basis of a trajectory inputby a user in order to surround a moving object in a video content, atarget region within a picture contained in a video content can beproperly determined.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a functional configuration of a dataprocessing apparatus according to Embodiment 1 of the present invention.

FIG. 2 is a flowchart showing a flow of a whole processing by the dataprocessing apparatus in Embodiment 1 according to the present invention.

FIG. 3 is a flowchart showing a detailed processing flow of an inputprocess in Embodiment 1 according to the present invention.

FIG. 4 is a drawing showing an example of a trajectory expressed byinput data in Embodiment 1 according to the present invention.

FIG. 5 is a flowchart showing a detailed processing flow of a movementcalculation process in Embodiment 1 according to the present invention.

FIG. 6 is a drawing for illustrating a pixel group to be selected inEmbodiment 1 according to the present invention.

FIG. 7 is a drawing showing an example of a result of calculation of anamount of movement and moving direction of the pixel group in Embodiment1 according to the present invention.

FIG. 8 is a drawing showing an example of a result of calculation of anamount of movement and moving direction of the pixel group in Embodiment1 according to the present invention.

FIG. 9 is a drawing for illustrating a correction processing on theinput region performed by a correction unit in Embodiment 1 according tothe present invention.

FIG. 10 is a drawing showing an example of a trajectory expressed byinput data in Modification 1 of Embodiment 1 according to the presentinvention.

FIG. 11 is a drawing showing an example of a pixel group to be selectedin Modification 1 of Embodiment 1 according to the present invention.

FIG. 12 is a drawing showing an example of an input region specified inModification 2 of Embodiment 1 according to the present invention.

FIG. 13 is a drawing showing an example of a pixel group to be selectedin Modification 2 of Embodiment 1 according to the present invention.

FIG. 14 is a flowchart showing a detailed processing flow of a movementcalculation process in Modification 3 of Embodiment 1 according to thepresent invention.

FIG. 15 is a drawing showing an example of a pixel group to be selectedin Modification 3 of Embodiment 1 according to the present invention.

FIG. 16 is a drawing showing an example of a result of calculation of anamount of movement and moving direction of the pixel group inModification 3 of Embodiment 1 according to the present invention.

FIG. 17 is a drawing showing an example of a result of calculation of anamount of movement and moving direction of the pixel group inModification 3 of Embodiment 1 according to the present invention.

FIG. 18 is a drawing for illustrating a correction processing on aninput region performed by a correction unit in Modification 4 ofEmbodiment 1 according to the present invention.

FIG. 19 is a drawing for illustrating a correction processing on aninput region performed by a correction unit in Modification 5 ofEmbodiment 1 according to the present invention.

FIG. 20 is a drawing for illustrating a correction processing on aninput region performed by a correction unit in Modification 6 ofEmbodiment 1 according to the present invention.

FIG. 21 is a drawing for illustrating a correction processing on aninput region performed by a correction unit in Modification 7 ofEmbodiment 1 according to the present invention.

FIG. 22 is a block diagram showing a functional configuration of a dataprocessing apparatus according to Embodiment 2 of the present invention.

FIG. 23 is a flowchart showing a detailed processing flow of a movementcalculation process in Embodiment 2 according to the present invention.

FIG. 24 is a drawing for illustrating a pixel group selected inEmbodiment 2 according to the present invention.

FIG. 25 is a drawing for illustrating a processing of calculating anamount of movement and moving direction of an image within an inputregion in Embodiment 2 according to the present invention.

FIG. 26 is a drawing for illustrating a processing of calculating anamount of movement and moving direction of an image within an inputregion in Embodiment 2 according to the present invention.

FIG. 27 is a drawing for illustrating a processing of correcting aninput region in Embodiment 2 according to the present invention.

FIG. 28 is a block diagram showing a functional configuration of a dataprocessing apparatus according to Embodiment 3 of the present invention.

FIG. 29 is a flowchart showing a flow of a whole processing by the dataprocessing apparatus in Embodiment 3 according to the present invention.

FIG. 30 is a drawing showing an example of a trajectory expressed byinput data in Embodiment 3 according to the present invention.

FIG. 31 is a flowchart showing a detailed processing flow of acorrection process in Embodiment 3 according to the present invention.

FIG. 32 is a drawing for illustrating a processing of selecting apicture for correcting an input region in Embodiment 3 according to thepresent invention.

FIG. 33A is a drawing showing an example of a method for correcting aninput region in Embodiment 3 according to the present invention.

FIG. 33B is a drawing showing an example of a method for correcting aninput region in Embodiment 3 according to the present invention.

FIG. 33C is a drawing showing an example of a method for correcting aninput region in Embodiment 3 according to the present invention.

FIG. 34 is a block diagram showing a functional configuration of a dataprocessing apparatus according to Embodiment 4 of the present invention.

FIG. 35 is a flowchart showing a detailed processing flow of acorrection process in Embodiment 4 according to the present invention.

FIG. 36 is a drawing showing an example of a target region surrounded bya complemented trajectory in Embodiment 4 according to the presentinvention.

FIG. 37 is a block diagram showing a functional configuration of a dataprocessing apparatus according to Embodiment 5 of the present invention.

FIG. 38 is a flowchart showing a detailed processing flow of acorrection process in Embodiment 5 according to the present invention.

FIG. 39 is a drawing for illustrating an example of a method fordetermining an algorithm in Embodiment 5 according to the presentinvention.

FIG. 40 is a drawing for illustrating an example of other method fordetermining an algorithm in Embodiment 5 according to the presentinvention.

FIG. 41 is a drawing for illustrating an example of further other methodfor determining an algorithm in Embodiment 5 according to the presentinvention.

FIG. 42 is a block diagram showing a functional configuration of a dataprocessing apparatus according to one embodiment of the presentinvention.

DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments according to the present invention will bedescribed with reference to the drawings.

Embodiment 1

A data processing apparatus 100 according to Embodiment 1 of the presentinvention determines a target region within a picture contained in avideo content on the basis of a trajectory input by a user in order tosurround a moving object in the video content. Particularly, in thepresent embodiment, after the user completes the input, the dataprocessing apparatus 100 corrects the input region using the amount ofmovement and moving direction of an image within an input region, whichis a region specified by the input trajectory, to thereby determine atarget region.

The picture contained in the video content means one of pictures thatform the video. The image within the input region means an image formedby pixels contained in the input region, which is a region within thepicture. The object means an object presented in the video content (forexample, people, animals, vehicles, and marks).

Hereinafter, the data processing apparatus according to Embodiment 1 ofthe present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing the functional configuration of thedata processing apparatus 100 according to Embodiment 1 of the presentinvention.

[Internal Configuration of Data Processing Apparatus]

The data processing apparatus 100 includes a video content obtainingunit 110, a decoding unit 120, a display unit 160, a movementcalculation unit 140, a correction unit 150, and an input unit 130.

An input means 190 is a specific input means with which the user inputsthe target region into the data processing apparatus 100, such as aremote control, a mouse, or a keyboard. In the case where the dataprocessing apparatus 100 includes a touch screen, the input means 190may be a stylus such as a touch pen, or a finger of the user. The inputmeans 190 may be an apparatus that enables operation by a gesture orvoice.

The video content obtaining unit 110 obtains a video content to beviewed by the user from broadcasting, a network, or a secondary storageunit, for example.

The decoding unit 120 decodes the video content obtained by the videocontent obtaining unit 110.

The input unit 130 receives the input data from the user via the inputmeans 190. Specifically, the input unit 130 receives the input datashowing the trajectory input by the user in the displayed video content.In the present embodiment, the input data is the data showing thetrajectory input by the user across several pictures contained in thevideo content.

The movement calculation unit 140 calculates the amount of movement andmoving direction of the image within the input region. Namely, themovement calculation unit 140 calculates the amount of movement andmoving direction of the image within the input region. In the presentembodiment, the movement calculation unit 140 calculates the amount ofmovement and moving direction of the pixel group contained in the inputregion, to thereby calculate the amount of movement and moving directionof the image within the input region. A method for calculating theamount of movement and moving direction will be described in detaillater.

The input region is a region specified by the trajectory expressed bythe input data. In the present embodiment, the input region is a regionsurrounded by the trajectory expressed by the input data. The pixelgroup means a group of pixels composed of a predetermined number ofpixels.

The correction unit 150 corrects the input region using the amount ofmovement and moving direction of the image within the input regioncalculated by the movement calculation unit 140, to thereby determinethe target region. Then, the correction unit 150 outputs thethus-determined target region to the display unit 160. Alternatively,the correction unit 150 may output the thus-determined target region toan external picture retrieving apparatus, for example. A method forcorrecting the input region will be described in detail later.

The display unit 160 displays the video content decoded by the decodingunit 120 and the trajectory expressed by the input data and received bythe input unit 130. Further, the display unit displays the informationshowing the target region determined by the correction unit 150.

[Whole Operation of Data Processing Apparatus]

Next, a variety of operations in the thus-configured data processingapparatus 100 will be described.

FIG. 2 is a flowchart showing a flow of a whole processing by the dataprocessing apparatus 100 according to Embodiment 1 of the presentinvention. The data processing apparatus 100 executes an input processof receiving the input data (Step S210), a movement calculation processof calculating the amount of movement and moving direction of the pixelgroup within the input region (Step S220), and a correction process ofcorrecting the input region (Step S230).

[Input Process]

First, the input process (Step S210) will be described in detail. FIG. 3is a flowchart showing a detailed processing flow of the input process(Step S210) in Embodiment 1 of the present invention.

When the user starts input of the trajectory using the input means 190,the input unit 130 of the data processing apparatus 100 receives theinput data showing the input trajectory. Subsequently, the input unit130 transmits the received input data to the movement calculation unit140 and to the display unit 160 (Step S310). While the user continuesthe input, the input unit 130 continuously transmits the input data tothe movement calculation unit 140 and to the display unit 160 (StepS320). The display unit 160 displays the received input data as it iswith the input data being overlaid on the video content.

FIG. 4 is a drawing showing an example of a trajectory expressed by theinput data in Embodiment 1 according to the present invention.Specifically, FIG. 4 is a drawing showing an example of a process inwhich the user inputs the trajectory across several pictures in order tosurround the moving object in the video content.

In the three pictures (pictures X, X+1, and X+2) contained in the videocontent, the object 420 moves from left to right in the horizontaldirection. The user inputs the trajectories 410, 411, and 412 so as tosurround the object 420. Finally, following the movement of the object,the user unintendedly inputs an elliptical trajectory 412 that encirclesa region considerably larger than the object.

[Movement Calculation Process]

Next, the movement calculation process (Step S220) will be described indetail. FIG. 5 is a flowchart showing a detailed processing flow of themovement calculation process (Step S220) in Embodiment 1 of the presentinvention.

[Recording Picture when Input is Started/Completed]

When the movement calculation unit 140 starts receiving the input datafrom the input unit 130, the movement calculation unit 140 refers to thevideo content received from the video content obtaining unit 110concurrently, and records the ID of the picture displayed at a timingwhen the user starts the input (hereinafter, referred to as the “initialpicture”) (Step S510). Next, when the movement calculation unit 140completes receiving the input data from the input unit 130 (Step S520),the movement calculation unit 140 refers to the video content receivedfrom the video content obtaining unit 110 concurrently, and records theID of the picture displayed at a timing when the user completes theinput (hereinafter, referred to as the “final picture”) (Step S530).

[Selection of Pixel Group]

Next, the movement calculation unit 140 selects the pixel group forcalculating the amount of movement and moving direction of the imagewithin the input region (Step S540). Specifically, the movementcalculation unit 140 specifies the input region on the basis of thetrajectory expressed by the input data. Here, as the input region, themovement calculation unit 140 specifies the region surrounded by thetrajectory expressed by the input data. Moreover, the movementcalculation unit 140 selects the pixel group contained in the inputregion in all the pictures from the initial picture to the finalpicture. Here, the pixel group contained in the input region includes apixel group in which at least part of the pixels thereof is contained inthe input region.

FIG. 6 is a drawing for illustrating a pixel group to be selected inEmbodiment 1 of the present invention. As shown in FIG. 6, in each ofthe pictures X, X+1, and X+2, the movement calculation unit 140 selectsthe pixel group even part of which is contained in the input region (thehatched pixel group) from a plurality of pixel groups contained in thepicture.

[Calculation of Amount of Movement and Moving Direction]

Next, the movement calculation unit 140 calculates the amount ofmovement and moving direction of the pixel group selected in Step S540,to thereby calculate the amount of movement and moving direction of theimage within the input region (Step S550).

The amount of movement and moving direction of the pixel group selectedin a picture is calculated as a relative amount of movement and movingdirection with respect to the picture immediately before the picture.Accordingly, for example, in the case where the trajectory is input inthe pictures N to N+m, the amount of movement and moving direction ofthe pixel group is calculated in the pictures N+1 to N+m.

For example, the amount of movement and moving direction of the pixelgroup to be calculated in FIG. 6 is the amount of movement and movingdirection between the picture X+1 and the picture X and the amount ofmovement and moving direction between the picture X+2 and the pictureX+1.

FIG. 7 is a drawing showing an example of a result of calculation of theamount of movement and moving direction of the pixel group in Embodiment1 according to the present invention. Only pixel groups A to I move inthe pictures X and X+1. For example, the pixel group A0 in the picture Xmoves to the pixel group A1 in the picture X+1. Accordingly, the amountof movement and moving direction of the pixel group A1 to the pixelgroup A0 is 5 pixel groups in the x direction and 0 pixel groups in they direction. Similarly, the amounts of movement and moving directions ofthe pixel groups B1 to I1 to the pixel groups B0 to I0, respectively,are 5 pixel groups in the x direction and 0 pixel groups in the ydirection. The amount of movement of other pixel groups is 0.

FIG. 8 is a drawing showing an example of a result of calculation of anamount of movement and moving direction of the pixel group in Embodiment1 according to the present invention. Similarly to the case of theamount of movement and moving direction in the picture X+1 relative tothe picture X, the amounts of movement and moving directions of thepixel groups A2 to I2 to the pixel groups A1 to I1, respectively, are 5pixel groups in the x direction and 0 pixel groups in the y direction.The amount of movement of other pixel groups is 0.

The movement calculation unit 140 calculates the averages of thethus-calculated amounts of movement and moving directions of the pixelgroups for each of the pictures. Moreover, the movement calculation unit140 sums the averages calculated for the respective pictures as theamount of movement and moving direction of the image within the inputregion in the pictures X to X+2.

Here, a method for calculating the averages of the amounts of movementand moving directions of the pixel groups for each of the pictures andsumming the averages will be described in detail.

First, the movement calculation unit 140 calculates the averages of theamounts of movement and moving directions of the pixel groups for eachof the pictures. At this time, the movement calculation unit 140neglects the pixel groups not moved, and derives the averages. Namely,the movement calculation unit 140 calculates the averages of the amountsof movement and moving directions of the pixel groups whose amount ofmovement is greater than 0 for each of the pictures.

Next, the movement calculation unit 140 sums the averages calculated forthe respective pictures as the amount of movement and moving directionof the image within the input region. For example, in FIG. 7 and FIG. 8,the amount of movement and moving direction of the image within theinput region is calculated as follows.

Average in the picture X+1=(amount of movement and moving direction ofthe pixel group A1 (5,0)+amount of movement and moving direction of thepixel group B1 (5,0)+amount of movement and moving direction of thepixel group C1 (5,0)+amount of movement and moving direction of thepixel group D1 (5,0)+amount of movement and moving direction of thepixel group E1 (5,0)+amount of movement and moving direction of thepixel group F1 (5,0)+amount of movement and moving direction of thepixel group G1 (5,0)+amount of movement and moving direction of thepixel group H1 (5,0)+amount of movement and moving direction of thepixel group I1. (5,0))/9=(5,0)

Average in the picture X+2=(amount of movement and moving direction ofthe pixel group A2 (5,0)+amount of movement and moving direction of thepixel group B2 (5,0)+amount of movement and moving direction of thepixel group C2 (5,0)+amount of movement and moving direction of thepixel group D2 (5,0)+amount of movement and moving direction of thepixel group E2 (5,0)+amount of movement and moving direction of thepixel group F2 (5,0)+amount of movement and moving direction of thepixel group G2 (5,0)+amount of movement and moving direction of thepixel group H2 (5,0)+amount of movement and moving direction of thepixel group I2 (5,0))/9=(5,0)

Amount of movement and moving direction of the image within the inputregion=average in the picture X+1 (5,0)+average in the picture X+2(5,0)=(10,0)

The movement calculation unit 140 does not always need to calculate theamount of movement and moving direction of the image within the inputregion by the method as described above. For example, the movementcalculation unit 140 may calculate the sum of the averages of theamounts of movement and moving directions of all the pixel groups(including the pixel groups not moved) contained in the input region asthe amount of movement and moving direction of the image within theinput region.

[Correction Process]

Next, the correction process. (Step S230) will be described in detail.As the target region, the correction unit 150 determines theintersection of a region after movement obtained by moving the inputregion according to the calculated amount of movement and movingdirection and the input region. Specifically, the correction unit 150moves the input region by the amount of movement in the moving directionon the final picture of the several pictures to which the trajectory isinput. Subsequently, the correction unit 150 determines the intersectionof the region before movement and that after movement as the targetregion.

FIG. 9 is a drawing for illustrating a correction processing on theinput region performed by the correction unit 150 in Embodiment 1according to the present invention. The correction unit 150 calculatesthe region after movement 910 obtained by moving the input region 900 inthe calculated moving direction by the calculated amount of movement,i.e., in the x direction by 10 pixel groups. Next, the correction unit150 determines the intersection of the input region 900 and the regionafter movement 910 as the target region 920.

As above, the data processing apparatus 100 according to the presentembodiment can correct the input region specified by the trajectoryinput by the user in order to surround the object, to thereby obtain thetarget region that the user originally intended to surround.

The data processing apparatus 100 according to the present embodimentcan also correct the input region using the amount of movement andmoving direction of the image within the input region. Usually, theinput region contains the object that the user intends to surround.Accordingly, the data processing apparatus 100 can correct the inputregion according to the movement of the object. Thereby, even if themovement of the object causes the input of the trajectory to be out oforder, the input region can be corrected using the movement, and thetarget region can be properly determined. Namely, the region that theuser intends to surround, i.e., the region containing the object can beproperly determined as the target region.

Further, according to the amount of movement and moving direction of theimage within the input region, as the target region, the data processingapparatus 100 according to the present embodiment can determine theintersection of the region after movement obtained by moving the inputregion and the input region. Accordingly, even if the input regionbecomes larger than necessary because the user follows the movement ofthe object, the region that contains the object and is smaller than theinput region can be determined as the target region. As a result, thetarget region can be properly determined.

Moreover, if the target region thus determined is output to a pictureretrieving apparatus or the like, the picture content related to theobject that the user intends to designate (a still picture content or avideo content) can be efficiently retrieved.

Next, modifications of Embodiment 1 will be described below. In thedescription of each modification below, differences from Embodiment 1will be mainly described, and the illustration and description ofsimilarities to Embodiment 1 will be properly omitted.

(Modification 1 of Embodiment 1)

Unlike Embodiment 1, Modification 1 of Embodiment 1 includes otherprocessing of specifying the input region from the input trajectory andselecting the pixel group contained in the input region. At the time ofinputting the trajectory using the input means 190, the user does notalways input the elliptical trajectory as shown in FIG. 4. For example,it is thought that the trajectory as shown in FIG. 10 may be input.Accordingly, in the present modification, the movement calculation unit140 approximates the trajectory expressed by the input data to anellipse, and specifies the region surrounded by the ellipse as the inputregion. Hereinafter, the data processing apparatus according to thepresent modification will be described with reference to the drawings.

FIG. 10 is a drawing showing an example of a trajectory expressed by theinput data in Modification 1 of Embodiment 1 according to the presentinvention. Specifically, FIG. 10 is a drawing showing other example of aprocess to which the user inputs the trajectory across several picturesso as to surround the moving object in the video content.

In the three pictures (pictures X, X+1, and X+2) contained in the videocontent, the object moves from left to right in the horizontal directionin the same manner as in the case of FIG. 4. Initially, the user inputsthe trajectory following the movement of the object. Finally, however,the user inputs the trajectory that encircles the object in the pictureX+2. In this case, the movement calculation unit 140 has difficulties tospecify the input region from the trajectory expressed by the input datain a simple manner.

Accordingly, the movement calculation unit 140 approximates thetrajectory expressed by the input data to an ellipse so as to surroundthe entire trajectory expressed by the input data, and specifies theregion surrounded by the ellipse as the input region. The movementcalculation unit 140 may approximate the trajectory to an ellipse usingan ordinary method such as the least squares method.

FIG. 11 is a drawing showing an example of a pixel group to be selectedin Modification 1 of Embodiment 1 according to the present invention. Inthe present modification, as shown in FIG. 11, from a plurality of pixelgroups contained in the picture, the movement calculation unit 140selects the pixel group contained in the input region surrounded by theellipse obtained by approximating the trajectory input by the user.

As above, the data processing apparatus 100 according to the presentmodification can properly determine the target region even if thetrajectory input by the user is not an ellipse.

(Modification 2 of Embodiment 1)

Unlike Embodiment 1, Modification 2 of Embodiment 1 also includes otherprocessing of specifying the input region from the input trajectory andselecting the pixel group contained in the input region, as well as inModification 1 above. Specifically, in the present modification, as theinput region, the movement calculation unit 140 specifies therectangular region containing the trajectory expressed by the inputdata. Hereinafter, the data processing apparatus in the presentmodification will be described with reference to the drawings.

FIG. 12 is a drawing showing an example of an input region specified inModification 2 of Embodiment 1 according to the present invention. As inFIG. 12, in the case where a coordinate frame is set in the picture, themovement calculation unit 140 calculates the minimum value and maximumvalue in the x coordinate and the minimum value and maximum value in they coordinate in the trajectory expressed by the input data. As the inputregion, the movement calculation unit 140 specifies the rectangularregion having four vertices in combination of the minimum value andmaximum value in the x coordinate and the minimum value and maximumvalue in the y coordinate thus calculated.

FIG. 13 is a drawing showing an example of a pixel group to be selectedin Modification 2 of Embodiment 1 according to the present invention. Inthe present modification, as shown in FIG. 13, from a plurality of pixelgroups contained in the picture, the movement calculation unit 140selects the pixel group contained in the rectangular input regionspecified by the trajectory input by the user.

As above, the data processing apparatus 100 according to the presentmodification can properly determine the target region even if the inputregion is not elliptical.

(Modification 3 of Embodiment 1)

Unlike Embodiment 1, Modification 3 of Embodiment 1 includes otherprocessing of selecting the pixel group in order to calculate the amountof movement and moving direction of the image within the input region.Specifically, in the present modification, on the basis of the inputtrajectory, the movement calculation unit 140 selects at least one pixelgroup in at least initial one picture among several pictures to which atrajectory is input. Hereinafter, the data processing apparatusaccording to the present modification will be described with referenceto the drawings.

FIG. 14 is a flowchart showing a detailed processing flow of a movementcalculation process in Modification 3 of Embodiment 1 according to thepresent invention. In FIG. 14, same reference numerals will be given tothe same processings as those in FIG. 5, and the description thereofwill be omitted.

On the basis of the input trajectory, the movement calculation unit 140selects at least one pixel group in at least initial one picture amongseveral pictures to which the trajectory is input (S1440). The specificexample thereof will be described in detail using FIG. 15 to FIG. 17.

FIG. 15 is a drawing showing an example of a pixel group to be selectedin Modification 3 of Embodiment 1 according to the present invention.Specifically, FIG. 15 shows a pixel group selected in the initialpicture X among the pictures X to X+2 to which the trajectory is input.

As shown in FIG. 15, the trajectory input by the user in the picture Xis a half input trajectory, and does not form a closed region.Accordingly, the movement calculation unit 140 selects pixel groups 1520contained in a region surrounded by a line segment 1500 extending fromone end point of the input trajectory in the vertical direction, a linesegment 1510 extending from the other end point of the input trajectoryin the horizontal direction, and the input trajectory. As shown in FIG.16 and FIG. 17, the movement calculation unit 140 calculates the amountsof movement and moving directions of the thus-selected pixel groups 1520in the pictures X to X+2 as the amount of movement and moving directionof the image within the input region.

The movement calculation unit 140 does not always need to select thepixel group in the initial one picture on the basis of the inputtrajectory. For example, the movement calculation unit 140 may selectthe pixel group in the initial two or more pictures on the basis of theinput trajectory.

The movement calculation unit 140 does not always need to calculate theamount of movement and moving direction of the pixel group as describedabove. For example, the movement calculation unit 140 may calculate a“motion vector” in a macroblock (8×8 pixels) unit using a picturecompression method such as the Moving Picture Experts Group (MPEG) asthe amount of movement and moving direction of the pixel group of 8×8pixels.

As above, the data processing apparatus 100 according to the presentmodification can select at least one pixel group in at least initial onepicture among several pictures on the basis of the input trajectory.Accordingly, the pixel group can be selected on the basis of the inputtrajectory before the object moves largely. If the amount of movementand moving direction of the image within the input region is calculatedusing the pixel group thus selected, the amount of movement and movingdirection thereof can be calculated according to the movement of theobject. Accordingly, if the input region is corrected using the amountof movement and moving direction of the image within the input regionthus calculated, the target region can be properly determined.

(Modification 4 of Embodiment 1)

Unlike Embodiment 1, Modification 4 of Embodiment 1 includes otherprocessing of correcting the input region to thereby determine thetarget region. Specifically, in the present modification, as the targetregion, the correction unit 150 determines the intersection of theregion after movement obtained by moving the input region in thedirection opposite to the moving direction of the image within the inputregion by the amount of movement and the input region in the initialpicture among several pictures. Hereinafter, the data processingapparatus according to the present modification will be described withreference to the drawings.

FIG. 18 is a drawing for illustrating a correction processing on theinput region performed by the correction unit 150 in Modification 4 ofEmbodiment 1 according to the present invention. As shown in FIG. 18,the correction unit 150 calculates a region after movement 1810 obtainedby moving an input region 1800 in the direction opposite to thecalculated moving direction by the calculated amount of movement in theinitial picture X. Next, the correction unit 150 determines theintersection of the input region 1800 and the region after movement 1810as a target region 1820.

As above, the data processing apparatus 100 according to the presentmodification can provide the same effect as that in Embodiment 1.

(Modification 5 of Embodiment 1)

Unlike Embodiment 1, Modification 5 of Embodiment 1 includes otherprocessing of correcting the input region to thereby determine thetarget region. Specifically, in the present modification, the correctionunit 150 compresses the input region according to the amount of movementand moving direction of the image within the input region to therebydetermine the target region. Hereinafter, the data processing apparatusaccording to the present modification will be described with referenceto the drawings.

FIG. 19 is a drawing for illustrating a correction processing on theinput region performed by the correction unit 150 in Modification 5 ofEmbodiment 1 according to the present invention. As shown in FIG. 19,the correction unit 150 compresses the input region according to thecalculated amount of movement and moving direction to thereby determinethe target region. Specifically, the correction unit 150 compresses theinput region in the calculated moving direction by the calculated amountof movement to thereby determine the target region in the final pictureamong several pictures to which the user inputs the trajectory.

More specifically, first, the correction unit 150 sets an x′-axis 1910parallel to the calculated moving direction in the final picture X+2.Next, among all the points that form a trajectory 1920 input by theuser, the correction unit 150 specifies a point 1930 located at theleftmost on the x′-axis, a point 1940 located at the rightmost, and apoint 1950 obtained by moving the point 1930 in the calculated movingdirection by the calculated amount of movement. Next, the correctionunit 150 calculates a distance m in the x′-axis direction between thepoint 1930 and the point 1950 and a distance n in the x′-axis directionbetween the point 1940 and the point 1950. Finally, wherein the x′coordinate of the point 1940 is 0, the correction unit 150 compressesthe x′ coordinate of the points except the point 1940 among all thepoints that form the trajectory 1920 to n/(m+n). Thereby, the trajectory1920 input by the user is compressed into a trajectory 1960. Thus, thecorrection unit 150 compresses the input region 1900 to therebydetermine the target region 1970.

As above, the data processing apparatus 100 according to the presentmodification can compress the input region according to the amount ofmovement and moving direction of the image within the input region tothereby determine the target region. Accordingly, even if the inputregion becomes larger than necessary because the user follows themovement of the object, the region that contains the object and issmaller than the input region can be determined as the target region. Asa result, the target region can be properly determined.

In the present modification, the correction unit 150 determines thetarget region in the final picture among the several pictures to whichthe user inputs the trajectory. The correction unit 150, however, doesnot always to thereby determine the target region in the final picture.For example, the correction unit 150 may determine the target region inthe picture immediately before the final picture. For example, thecorrection unit 150 may also specify the picture when the trajectory forsurrounding the object starts to be closed, and determine the targetregion in the specified picture.

(Modification 6 of Embodiment 1)

Unlike Embodiment 1, Modification 6 of Embodiment 1 includes otherprocessing of correcting the input region to thereby determine thetarget region as well as in Modification 5. Specifically, in the presentmodification, the correction unit 150 compresses the input region in thedirection opposite to the moving direction by the amount of movement tothereby determine the target region in the initial picture among severalpictures. Hereinafter, the data processing apparatus according to thepresent modification will be described with reference to the drawings.

FIG. 20 is a drawing for illustrating a correction processing on theinput region performed by the correction unit 150 in Modification 6 ofEmbodiment 1 according to the present invention. As shown in FIG. 20,the correction unit 150 compresses the input region in the directionopposite to the calculated moving direction by the calculated amount ofmovement to thereby determine the target region in the initial pictureamong the several pictures to which the user inputs the trajectory.

More specifically, first, the correction unit 150 sets an x′-axis 2010parallel to the calculated moving direction in the initial picture X.Next, among all the points that form a trajectory 2020 input by theuser, the correction unit 150 specifies a point 2030 located at theleftmost in the x′-axis, a point 2040 located at the rightmost, and apoint 2050 obtained by moving the point 2040 in the direction oppositeto the calculated moving direction by the calculated amount of movement.Next, the correction unit 150 calculates a distance m in the x′-axisdirection between the point 2030 and the point 2050 and a distance n inthe x′-axis direction between the point 2040 and the point 2050.Finally, wherein the x′ coordinate of the point 2030 is 0, thecorrection unit 150 compresses the x′ coordinate of the points exceptthe point 2040 among all the points that form the trajectory 2020 tom/(m+n). Thereby, the trajectory 2020 input by the user is compressedinto a trajectory 2060. Thus, the correction unit 150 compresses theinput region 2000 to thereby determine the target region.

As above, the data processing apparatus 100 according to the presentmodification can provide the same effect as that in Modification 6.

(Modification 7 of Embodiment 1)

Unlike Embodiment 1, Modification 7 of Embodiment 1 includes otherprocessing of correcting the input region to thereby determine thetarget region as well as in Modifications 5 and 6. In the presentmodification, the input region has a rectangular shape as inModification 2. Hereinafter, the data processing apparatus according tothe present modification will be described with reference to thedrawings.

FIG. 21 is a drawing for illustrating a correction processing on theinput region performed by the correction unit 150 in Modification 7 ofEmbodiment 1 according to the present invention. As shown in FIG. 21,the correction unit 150 compresses the input region in the calculatedmoving direction by the calculated amount of movement to therebydetermine the target region in the initial picture among the pictures towhich the user inputs the trajectory.

Specifically, first, the correction unit 150 sets the top side of theinput region 2100 as an x-axis 2120, the right side thereof as a y-axis2130, and a point of intersection 2140 between the top side and theright side as the origin in the final picture X+2. Next, the correctionunit 150 specifies a point 2160 obtained by moving a vertex 2150 on thelower left of the input region 2100 in the calculated moving directionby the calculated amount of movement. Next, the correction unit 150calculates a distance m in the x-axis direction between the point 2150and the point 2160 and a distance n in the x-axis direction between thepoint 2150 and the point 2140. The correction unit 150 also calculates adistance q in the y-axis direction between the point 2150 and the point2160 and a distance p in the y-axis direction between the point 2150 andthe point 2140. Finally, the correction unit 150 compresses the xcoordinate of all the points that form the trajectory 2110 to n/(m+n),and the y coordinate to p/(p+q). Thereby, the trajectory 2110 iscompressed into a trajectory 2170. Thus, the correction unit 150compresses the input region 2100 to thereby determine a target region2180.

As above, the data processing apparatus 100 according to the presentmodification can properly determine the target region even if the inputregion has a rectangular shape, as in Modification 5 or the like.

As above, Modifications 1 to 7 of Embodiment 1 according to the presentinvention have been described. Without contradiction, the respectivecomponents in Modifications 1 to 7 may be used in combination. Forexample, the data processing apparatus 100 may calculate the amount ofmovement and moving direction of the image within the input region as inModification 3, and determine the target region as in Modification 5 or6.

Embodiment 2

Next, Embodiment 2 according to the present invention will be described.Unlike Embodiment 1, the present embodiment includes other processing ofcalculating the amount of movement and moving direction of the imagewithin the input region and other processing of correcting the inputregion. Specifically, in the present embodiment, the movementcalculation unit calculates the amount of movement and moving directionof the image within the input region concurrently with the input by theuser, and the correction unit corrects the input region concurrentlywith the input by the user. Hereinafter, the data processing apparatusaccording to the present embodiment will be described with reference tothe drawings.

FIG. 22 is a block diagram showing a functional configuration of a dataprocessing apparatus 2200 according to Embodiment 2 of the presentinvention. In FIG. 22, same reference numerals will be given to the samecomponents as those in FIG. 1, and the description thereof will beomitted properly.

The data processing apparatus 2200 includes a video content obtainingunit 110, a decoding unit 120, a display unit 160, a movementcalculation unit 2240, and a correction unit 2250, and an input unit130.

The movement calculation unit 2240 calculates the amount of movement andmoving direction of the image within the input region. Specifically, themovement calculation unit 2240 calculates the amount of movement andmoving direction of the image within the input region concurrently withthe input by the user. In the present embodiment, the movementcalculation unit 2240 calculates the amount of movement and movingdirection of the image within the input region every time when thepicture to be displayed is switched. A method for calculating the amountof movement and moving direction of the image within the input regionwill be described in detail later.

The correction unit 2250 corrects the input region using the amount ofmovement and moving direction of the image within the input regioncalculated by the movement calculation unit 2240. Specifically, thecorrection unit 2250 corrects the input region concurrently with theinput by the user. In the present embodiment, the correction unit 2250corrects the input region every time when the picture to be displayed isswitched and the amount of movement and moving direction of the imagewithin the input region is calculated. A method for correcting the inputregion will be described in detail later.

Next, a variety of operations in the thus-configured data processingapparatus 2200 will be described. The flow of the entire processing bythe data processing apparatus 2200 in the present embodiment are thesame as that in FIG. 2 of Embodiment 1, and the illustration thereofwill be omitted.

A movement calculation process (Step S220) will be described in detail.FIG. 23 is a flowchart showing a detailed processing flow of themovement calculation process (Step S220) in Embodiment 2 according tothe present invention.

When the movement calculation unit 2240 starts receiving the input datafrom the input unit 130 (Yes in Step S2310), the movement calculationunit 2240 refers to the video content concurrently received from thevideo content obtaining unit 110, and records the ID of the picture(initial picture) displayed at a timing when the user starts input (StepS2320). Next, when the movement calculation unit 2240 completesreceiving the input data from the input unit 130 (Yes in Step S2330),the movement calculation unit 2240 refers to the video contentconcurrently received from the video content obtaining unit 110, andrecords the ID of the picture (final picture) displayed at a timing whenthe user completes the input (Step S2340).

If reception of the input data is not started (No in Step S2310) andreception of the input data is not completed (No in Step S2330), themovement calculation unit 2240 determines whether the picture displayedby the display unit 160 is at a timing of switching (Step S2350). Here,if the displayed picture is not at a timing of switching (No in StepS2350), the movement calculation unit 2240 returns to the processing inStep S2310.

On the other hand, if the displayed picture is at a timing of switching(Yes in Step S2350) or after the processing in Step S2340 is completed,the movement calculation unit 2240 selects the pixel group in order tocalculate the amount of movement and moving direction of the imagewithin the input region (Step S2360). The movement calculation unit 2240calculates the amount of movement and moving direction of the selectedpixel group (Step S2370). When calculation of the amount of movement andmoving direction of the pixel group is completed, the data processingapparatus 2200 immediately corrects the input region (Step S230).

Here, the processing of calculating the amount of movement and movingdirection of the image within the input region and the processing ofcorrecting the input region will be described in detail using FIG. 24 toFIG. 27.

FIG. 24 is a drawing for illustrating a pixel group to be selected inEmbodiment 2 according to the present invention. FIG. 25 and FIG. 26 area drawing for illustrating the processing of calculating the amount ofmovement and moving direction of the image within the input region inEmbodiment 2 according to the present invention. FIG. 27 is a drawingfor illustrating the processing of correcting the input region inEmbodiment 2 according to the present invention.

As shown in FIG. 24, the movement calculation unit 2240 specifies theinput region in the picture X+1. Here, the closed region is not formedonly by the trajectory input by the user until the picture X+1.Accordingly, the movement calculation unit 2240 complements thetrajectory input by the user until the picture X+1 as shown by thedashed line, and specifies the region surrounded by the complementedtrajectory as the input region. Subsequently, the movement calculationunit 2240 selects the pixel group (hatched pixel group) contained in theidentified input region. The method for complementing a trajectory is anexample. The trajectory may be complemented in any manner as long as thetrajectory is complemented so as to surround the object.

Next, as shown in FIG. 25, the movement calculation unit 2240 calculatesthe amount of movement and moving direction of the pixel group selectedin the picture X+1. While in FIG. 25, the selected pixel group isdifferent from that in FIG. 7 of Embodiment 1, the method forcalculating the amount of movement and moving direction of the pixelgroup is the same. Next, as shown in FIG. 27, the correction unit 2250corrects the input region. Specifically, the correction unit 2250corrects an input region 2730 surrounded by a trajectory 2700 input bythe user until the picture X+1, a line segment 2710 extending from theone end point of the trajectory 2700 in the horizontal direction, and aline segment 2720 extending from the other end point of the trajectory2700 in the vertical direction.

More specifically, first, the correction unit 2250 sets an x′-axis 2740parallel to the moving direction of the image within the input regioncalculated by the movement calculation unit 2240.

Next, the correction unit 2250 specifies a point 2750 located at theleftmost in the x′-axis direction, a point 2760 located at therightmost, and a point 2770 obtained by moving the point 2750 in thecalculated moving direction by the calculated amount of movement amongall the points that form the trajectory 2700. Next, the correction unit2250 calculates a distance m in the x′-axis direction between the point2750 and the point 2770 and a distance n in the x′-axis directionbetween the point 2760 and the point 2770. Finally, wherein the x′coordinate of the point 2760 is 0, the correction unit 2250 compressesthe x′ coordinate of all the points that form the trajectory 2700 ton/(m+n). Thereby, the trajectory 2700 input by the user is compressedinto a trajectory 2780. Thus, the correction unit 2250 compresses theinput region 2730 to thereby determine a target region 2790.

The displayed picture is switched from the picture X+1 to the pictureX+2. Then, in the picture X+2, the movement calculation unit 2240specifies the region surrounded by the trajectory corrected in thepicture X+1 and the trajectory input by the user in the displayedpicture X+2 as the input region, as shown in (c) of FIG. 24.Subsequently, the movement calculation unit 2240 selects the pixel groupwithin the specified input region. Subsequently, as shown in FIG. 26,the movement calculation unit 2240 calculates the amount of movement andmoving direction of the selected pixel group as the amount of movementand moving direction of the image within the input region.

The correction unit 2250 corrects the input region in the picture X+2 asin the same manner as in the picture X+1 to thereby determine the targetregion. The correction unit 2250 does not always need to correct theinput region in the picture X+2. Namely, the correction unit 2250 maydetermine the region corrected in the picture X+1 as the target region.

As above, the data processing apparatus 2200 according to the presentembodiment can correct the input region concurrently with the input bythe user. Accordingly, for example, the target region can be displayedin real time. This can assist the user to properly input the trajectory.

In the present embodiment, the correction unit 2250 corrects the inputregion every time when the displayed picture is switched. Namely, thecorrection unit 2250 corrects the input region for each picture, butdoes not always need to correct the input region for each picture. Forexample, the correction unit 2250 may correct the input region for everytwo or more pictures. The movement calculation unit 2240 may alsocalculate the amount of movement and moving direction of the imagewithin the input region for every two or more pictures.

Embodiment 3

Next, Embodiment 3 according to the present invention will be described.In the present embodiment, after the input by the user is completed, thecorrection unit corrects the input region on the basis of the trajectoryexpressed by the input data to thereby determine the target region.Namely, the data processing apparatus according to the presentembodiment determines the target region without using the amount ofmovement and moving direction of the image within the input region.Hereinafter, the data processing apparatus according to the presentembodiment will be described with reference to the drawings.

FIG. 28 is block diagram showing a functional configuration of a dataprocessing apparatus 2800 according to Embodiment 3 of the presentinvention. In FIG. 28, same reference numerals will be given to the samecomponents as those in FIG. 1, and the description will be omittedproperly.

The data processing apparatus 2800 includes a video content obtainingunit 110, a decoding unit 120, a display unit 160, a correction unit2850, and an input unit 130.

The correction unit 2850 corrects the input region. Specifically, afterthe input by the user across several pictures is completed, thecorrection unit 2850 corrects the input region on the basis of thetrajectory expressed by the input data to thereby determine the targetregion. In the present embodiment, the correction unit 2850 approximatesthe trajectory expressed by the input data to an ellipse. In the picturein which the point of intersection between the long axis of the ellipseand the trajectory is input, the correction unit 2850 determines aregion in the vicinity of the point of intersection as the targetregion.

Next, a variety of operations in the data processing apparatus 2800 willbe described. A variety of operations in the thus-configured dataprocessing apparatus 2800 will be described.

FIG. 29 is a flowchart showing a flow of a whole processing by the dataprocessing apparatus 2800 in Embodiment 3 according to the presentinvention. In FIG. 29, same reference numerals will be given to the sameprocessings as those in FIG. 2, and the description thereof will beomitted.

As shown in FIG. 29, the data processing apparatus 2800 executes aninput process of receiving the input data (Step S210) and a correctionprocess of correcting the input region (Step S2930).

FIG. 30 is a drawing showing an example of a trajectory expressed by theinput data in Embodiment 3 according to the present invention.Specifically, FIG. 30 is a drawing showing an example of a process towhich the user inputs the trajectory across several pictures in order tosurround a moving object in the video content.

In the four pictures (pictures X, X+1, X+2, and X+3) contained in thevideo content, an object 3020 moves from left to right in the horizontaldirection. The user inputs trajectories 3010, 3011, 3012, and 3013 inorder to surround the object 3020. Finally, the user unintendedly inputsan elliptical trajectory 3013 following the movement of the object 3020.

Here, a correction process (Step S2930) will be described in detail.FIG. 31 is a flowchart showing a detailed processing flow of acorrection process (Step S2930) in Embodiment 3 according to the presentinvention.

When the correction unit 2850 starts receiving the input data from theinput unit 130, the correction unit 2850 refers to the video contentconcurrently received from the video content obtaining unit 110, andrecords the ID of the picture (initial picture) displayed at a timingwhen the user starts the input (Step S510). Next, when the correctionunit 2850 completes reception of the input data from the input unit 130(Step S520), the correction unit 2850 refers to the video contentconcurrently received from the video content obtaining unit 110, andrecords the ID of the picture (final picture) displayed at a timing whenthe user completes the input (Step S530).

Next, among the several pictures to which the user inputs thetrajectory, the correction unit 2850 selects the picture for correctingthe input region (Step S3140). Specifically, the correction unit 2850approximates the trajectory expressed by the input data to an ellipseafter the user completes the input, and selects the picture in which thepoint of intersection between the long axis of the ellipse and thetrajectory is input.

FIG. 32 is a drawing for illustrating a picture for correcting the inputregion in Embodiment 3 according to the present invention. First, thecorrection unit 2850 calculates an ellipse 3210 to be finallyapproximated to a trajectory 3013 input by the user.

Subsequently, among two points of intersection between the calculatedlong axis of the ellipse 3220 and the trajectory 3013 input by the user,the correction unit 2850 specifies a point of intersection 3240 far fromthe start point 3230 of the trajectory 3013.

Next, as the picture for correcting the input region, the correctionunit 2850 selects the picture X+2 displayed at a timing when thespecified point of intersection 3240 is input.

Next, the correction unit 2850 corrects the input region in the selectedpicture (Step S3150). Specifically, the correction unit 2850 determinesthe region in the vicinity of the specified point of intersection as thetarget region in the selected picture.

FIG. 33A to FIG. 33C each are a drawing showing an example of a methodfor correcting the input region in Embodiment 3 according to the presentinvention.

For example, as shown in FIG. 33A, the correction unit 2850 determines aregion 3310 surrounded by a circle inscribed in the trajectory in thevicinity of the point of intersection 3240 as the target region in theselected picture X+2.

For example, as shown in FIG. 33B, in the selected picture X+2, as thetarget region, the correction unit 2850 determines a region surroundedby a circle 3330 whose diameter is as long as a short diameter 3320 ofan ellipse 3210 and which intersects the point of intersection 3240.

For example, as shown in FIG. 33C, in the selected picture X+2, thecorrection unit 2850 equally divides a long axis 3220 by the number ofpictures displayed from the initial picture X to the selected pictureX+2 (here, “3”). Of the regions surrounded by line segments 3340 and3350 that intersect the dividing point and perpendicularly intersect thelong axis 3220 and a trajectory 3013, the correction unit 2850determines a region 3360 containing the point of intersection 3240 asthe target region.

The correction unit 2850 may approximate the trajectory expressed by theinput data to an ellipse, and correct the input region as the region inthe vicinity of the point of intersection between the long axis of theellipse and the trajectory, and the correction method will not belimited to the method above.

As above, the data processing apparatus 2800 according to the presentembodiment can correct the input region on the basis of the shape of thetrajectory, and calculation of the amount of movement and movingdirection of the image within the input region is unnecessary.Accordingly, calculation load can be reduced.

Further, the data processing apparatus 2800 according to the presentembodiment can approximate the trajectory to an ellipse, and determinethe region in the vicinity of the point of intersection farthest fromthe start point in the picture in which the point of intersectionfarthest from the start point of the trajectory among the points ofintersection between the long axis of the ellipse and the trajectory isinput. Accordingly, the region in the vicinity of a position in whichthe trajectory for surrounding the object starts to be closed can bedetermined as the target region. Namely, because it is thought that theobject exists in the position in which the trajectory starts to beclosed, the target region containing the object can be properlydetermined.

Embodiment 4

Next, Embodiment 4 according to the present invention will be described.In the present embodiment, the correction unit complements thetrajectory expressed by the input data to thereby determine the targetregion. Namely, the data processing apparatus according to the presentembodiment determines the target region without using the amount ofmovement and moving direction of the image within the input region as inEmbodiment 3. Hereinafter, the data processing apparatus according tothe present embodiment will be described with reference to the drawings.

FIG. 34 is a block diagram showing a functional configuration of a dataprocessing apparatus 3400 according to Embodiment 4 of the presentinvention. In FIG. 34, same reference numerals will be given to the samecomponents as those in FIG. 1, and the description thereof will beomitted properly.

The data processing apparatus 3400 includes a video content obtainingunit 110, a decoding unit 120, a display unit 160, a correction unit3450, and an input unit 130.

The correction unit 3450 complements the trajectory expressed by theinput data to thereby determine the target region. In the presentembodiment, the correction unit 3450 approximates the trajectoryexpressed by the input data to an ellipse to thereby complement thetrajectory.

Next, a variety of operations in the thus-configured data processingapparatus 3400 will be described. A flow of the whole processing by thedata processing apparatus 3400 according to the present embodiment isthe same as that in FIG. 29 of Embodiment 3, and the illustrationthereof will be omitted.

FIG. 35 is a flowchart showing a detailed processing flow of thecorrection process (Step S2930) in Embodiment 4 according to the presentinvention.

The correction unit 3450 determines whether the trajectory input untilthe picture currently displayed can be complemented to form an ellipse(Step S3510). Namely, the correction unit 3450 determines whether thetrajectory can be approximated to an ellipse. Here, if the ellipsecannot be formed by complementation of the trajectory (No in StepS3510), the processing in Step S3510 is repeated.

On the other hand, if the trajectory can be complemented to form anellipse (Yes in Step S3510), as the target region, the correction unit3450 determines the region surrounded by the ellipse formed bycomplementation of the trajectory (Step S3520). Namely, the correctionunit 3450 approximates the trajectory expressed by the input data to anellipse to thereby complement the trajectory, and determines the regionsurrounded by the complemented trajectory as the target region.

FIG. 36 is a drawing showing an example of a target region surrounded bya complemented trajectory in Embodiment 4 according to the presentinvention. The correction unit 3450 adds a curved line 3610 to atrajectory 3600 expressed by the input data so as to provide an ellipseapproximated to the trajectory 3600. The correction unit 3450 determinesthe region surrounded by the trajectory 3600 and the curved line 3610(hatched region) as a target region 3620.

As above, the data processing apparatus 3400 according to the presentembodiment can complement the trajectory input by the user to therebydetermine the target region. Accordingly, the target region can beproperly determined before the object moves largely.

Further, the data processing apparatus 3400 according to the presentembodiment can approximate the trajectory input by the user to anellipse to thereby complement the trajectory. In the case where the userinputs a trajectory in order to surround a moving object, the shape ofthe trajectory is usually likely to be elliptical. Moreover, if theshape of the trajectory is elliptical, approximation can be performed atthe relatively small number of points. Accordingly, the trajectory inputby the user can be approximated to an ellipse to thereby properlydetermine the target region.

The correction unit 3450 does not always need to approximate thetrajectory to an ellipse. For example, if it is known in advance thatthe user inputs a rectangular trajectory or the like, the correctionunit 3450 may approximate the trajectory to a rectangle.

Embodiment 5

Next, Embodiment 5 according to the present invention will be described.In the present embodiment, on the basis of the input data, thecorrection unit determines an algorithm for determining the targetregion, and corrects the input region according to the determinedalgorithm to thereby determine the target region. Hereinafter, the dataprocessing apparatus according to the present embodiment will bedescribed with reference to the drawings.

FIG. 37 is a block diagram showing a functional configuration of a dataprocessing apparatus 3700 in Embodiment 5 according to the presentinvention. In FIG. 37, same reference numerals will be given to the samecomponents as those in FIG. 1, and the description thereof will beomitted properly.

The data processing apparatus 3700 includes a video content obtainingunit 110, a decoding unit 120, a display unit 160, a movementcalculation unit 140, a correction unit 3750, and an input unit 130.

On the basis of the input data, the correction unit 3750 determines analgorithm for determining the target region. The correction unit 3750corrects the input region according to the determined algorithm tothereby determine the target region.

Specifically, on the basis of the input data, the correction unit 3750determines whether to correct the input region using the amount ofmovement and moving direction of the image within the input region as inEmbodiment 1 or 2.

For example, on the basis of the shape of the trajectory expressed bythe input data, the correction unit 3750 determines whether to correctthe input region using the amount of movement and moving direction ofthe image within the input region. For example, on the basis offluctuation in the amount of movement and moving direction of the imagewithin the input region, the correction unit 3750 also determineswhether to correct the input region using the amount of movement andmoving direction of the amount of movement and moving direction of theimage within the input region.

Next, a variety of operations of the thus-configured data processingapparatus 3700 will be described. A flow of the whole processing by thedata processing apparatus 3700 according to the present embodiment isthe same as that in FIG. 2 of Embodiment 1, and the illustration thereofwill be omitted.

Here, a correction process different from that in Embodiment 1 (StepS230) will be described in detail. FIG. 38 is a flowchart showing adetailed processing flow of a correction process (Step S230) inEmbodiment 5 according to the present invention.

First, on the basis of the input data, the correction unit 3750determines an algorithm for determining the target region most properly(Step S3810). Subsequently, the correction unit 3750 corrects the inputregion according to the determined algorithm to thereby determine thetarget region (Step S3820).

Hereinafter, an example of a method for determining the algorithm willbe described using FIG. 39 to FIG. 41. FIG. 39 to FIG. 41 each are adrawing for illustrating an example of a method for determining thealgorithm in Embodiment 5 according to the present invention.

For example, as shown in FIG. 39, in the case where the several movingobjects exist in the input region, the correction unit 3750 determinesnot to correct the input region using the amount of movement and movingdirection of the image within the input region. Specifically, if thevalue showing fluctuation in the amount of movement and moving directionof the pixel group contained in the input region is not less than athreshold value, the correction unit 3750 determines not to correct theinput region using the amount of movement and moving direction of theimage within the input region. The value showing fluctuation is thevariance or standard deviation, for example.

Thus, if several moving objects exist within the input region, it isdifficult to calculate the amount of movement and moving direction ofthe object that the use tries to surround. Accordingly, the correctionunit 3750 can determine not to correct the input region using the amountof movement and moving direction of the image within the input region,to thereby properly determine the target region.

For example, as shown in FIG. 40, the correction unit 3750 also candetermine whether to approximate the trajectory expressed by the inputdata to an ellipse. For example, the correction unit 3750 determineswhether the value showing a difference between the ellipse obtained onthe basis of the trajectory expressed by the input data and thetrajectory is greater than the threshold value determined in advance, tothereby determine whether to approximate the trajectory to the ellipse.

Here, if the trajectory cannot be approximated to the ellipse, thecorrection unit 3750 determines to correct the input region using theamount of movement and moving direction of the image within the inputregion. Thereby, even if the method for complementing the trajectory asin Embodiment 4 cannot be used, the correction unit 3750 can properlydetermine the target region.

For example, as shown in FIG. 41, in the case where the movement of theobject is complex and the trajectory forms a distorted closed region,the correction unit 3750 determines not to correct the input regionusing the amount of movement and moving direction of the image withinthe input region. In such a case, the correction unit 3750 complementsthe trajectory to thereby determine the target region as in Embodiment4.

The method for determining the algorithm above is an example, and thecorrection unit 3750 may determine the algorithm on the basis of adifferent criterion.

As above, on the basis of the input data, the data processing apparatus3700 according to the present embodiment can properly determine thealgorithm for determining the target region, and can determine thetarget region more properly.

As above, the data processing apparatus according to embodiments of thepresent invention has been described on the basis of embodiments, butthe present invention will not be limited to these embodiments. Withoutdeparting from the scope of the present invention, the present inventionalso includes an embodiment with various modifications on the presentembodiment that are conceived by a person skilled in the art, or anembodiment in combination with components in a different embodiment or amodification thereof.

For example, in Embodiments 1 to 5 above, the data processing apparatusincludes the video content obtaining unit 110, the decoding unit 120,and the display unit 160, but does not always need to include thesecomponents. For example, as shown in FIG. 42, a data processingapparatus 10 may include an input unit 11 and a correction unit 12. Evenin this case, for example, the data processing apparatus 10 can providethe same effect as that in the embodiments above if the correction unit12 outputs a target region or an image within the target region to adisplay unit.

Moreover, in Embodiments 1 to 5, the display unit displays the targetregion determined by the correction unit, but does not always need todisplay the target region. In this case, for example, the correctionunit may transmit a target region or an image within the target regionto a picture retrieving apparatus.

Moreover, part or all of the components that the data processingapparatuses according to Embodiments 1 to 5 include may be composed ofone system LSI (Large Scale Integration). For example, the dataprocessing apparatus may be composed of a system LSI having an inputunit and a correction unit.

The system LSI is an ultra multifunctional LSI manufactured byintegrating a plurality of components on a chip, and specifically, is acomputer system including a microprocessor, a ROM (Read Only Memory), aRAM (Random Access Memory), and the like. The RAM stores a computerprogram. The microprocessor operates according to the computer program,and the system LSI achieves the function.

Here, while the LSI is referred to the system LSI, it may be referred toas an IC, an LSI, a super LSI, or an ultra LSI depending on the degreeof integration. Moreover, the method for integration is not limited tothe LSI, and integration may be achieved using a dedicated circuit or ageneral-purpose processor. After manufacturing the LSI, a programmableFPGA (Field Programmable Gate Array) or a reconfigurable processor inwhich connection or setting of the circuit within the LSI can bereconfigured may be used.

Further, if a technique of integration to replace the LSI appearsbecause of progress in the semiconductor techniques or other techniquesderived therefrom, the functional blocks may be naturally integratedusing such a technique. Examples thereof may include application ofbiotechnology.

Moreover, the present invention can be achieved as a data processingapparatus including the characteristic components as described above,but also can be achieved as a data processing method including theprocessings of the characteristic components included in the dataprocessing apparatus. Alternatively, the present invention can beachieved as a computer program causing a computer to execute each of thecharacteristic processings included in the data processing method. Sucha computer program can be distributed through a non-temporary computerreadable recording medium such as a CD-ROM and a communicating mediumsuch as the Internet.

INDUSTRIAL APPLICABILITY

The data processing apparatus according to one embodiment of the presentinvention is useful as a data processing apparatus that properlydetermines a target region within a picture contained in a video contenton the basis of a trajectory input by a user in order to surround amoving object in the video content. Further, the data processingapparatus according to one embodiment of the present invention is alsouseful as a video content retrieving apparatus or the like thatretrieves a video content different from the displayed video contentusing the determined target region.

REFERENCE SIGNS LIST

-   10, 100, 2200, 2800, 3400, 3700 Data processing apparatus-   11, 130 Input unit-   12, 150, 2250, 2850, 3450, 3750 Correction unit-   110 Moving picture content obtaining unit-   20 Decoding unit-   40, 2240 Movement calculation unit-   60 Display unit-   90 Input means

1. A data processing apparatus that determines a target region within apicture contained in a video content on the basis of a trajectory inputby a user in order to surround a moving object in the video content,said data processing apparatus comprising: an input unit configured toreceive input data showing the trajectory input by the user in the videocontent displayed; and a correction unit configured to correct an inputregion, which is a region specified by the trajectory expressed by theinput data, to thereby determine the target region.
 2. The dataprocessing apparatus according to claim 1, wherein the input data isdata showing the trajectory input by the user across several picturescontained in the video content, said data processing apparatus furthercomprises a movement calculation unit configured to calculate an amountof movement and moving direction of an image within the input region inthe several pictures, and said correction unit is configured to correctthe input region using the calculated amount of movement and movingdirection of the image within the input region, to thereby determine thetarget region.
 3. The data processing apparatus according to claim 2,wherein said correction unit is configured to correct the input regionafter the user completes the input.
 4. The data processing apparatusaccording to claim 3, wherein said correction unit is configured todetermine, as the target region, an intersection of a region aftermovement obtained by moving the input region according to the amount ofmovement and moving direction of the image within the input region andthe input region.
 5. The data processing apparatus according to claim 4,wherein said correction unit is configured to determine an intersectionof the region after movement obtained by moving the input region in themoving direction by the amount of movement and the input region as thetarget region in a final picture among the several pictures.
 6. The dataprocessing apparatus according to claim 4, wherein said correction unitis configured to determine an intersection of the region after movementobtained by moving the input region in a direction opposite to themoving direction by the amount of movement and the input region as thetarget region in an initial picture among the several pictures.
 7. Thedata processing apparatus according to claim 4, wherein said movementcalculation unit is configured to select at least one pixel group in atleast initial one picture among the several pictures on the basis of theinput trajectory, and calculate an amount of movement and movingdirection of the selected pixel group as the amount of movement andmoving direction of the image within the input region.
 8. The dataprocessing apparatus according to claim 3, wherein said correction unitis configured to compress the input region according to the amount ofmovement and the moving direction, to thereby determine the targetregion.
 9. The data processing apparatus according to claim 8, whereinsaid correction unit is configured to compress the input region in themoving direction by the amount of movement, to thereby determine thetarget region in the final picture among the several pictures.
 10. Thedata processing apparatus according to claim 8, wherein said correctionunit is configured to compress the input region in the directionopposite to the moving direction by the amount of movement, to therebydetermine the target region in the initial picture among the severalpictures.
 11. The data processing apparatus according to claim 8,wherein said movement calculation unit is configured to select at leastone pixel group in at least initial one picture among the severalpictures on the basis of the input trajectory, and calculate an amountof movement and moving direction of the selected pixel group as theamount of movement and moving direction of the image within the inputregion.
 12. The data processing apparatus according to claim 2, whereinsaid movement calculation unit is configured to calculate the amount ofmovement and moving direction of the image within the input regionconcurrently with the input, and said correction unit is configured tocorrect the input region concurrently with the input.
 13. The dataprocessing apparatus according to claim 1, wherein the input data isdata showing the trajectory input by the user across several picturescontained in the video content, and said correction unit is configuredto correct the input region on the basis of a shape of the trajectoryexpressed by the input data after the input across the several picturesis completed, to thereby determine the target region.
 14. The dataprocessing apparatus according to claim 13, wherein said correction unitis configured to approximate the trajectory expressed by the input datato an ellipse, and determine a region in the vicinity of a point ofintersection as the target region in a picture in which the point ofintersection is input, the point of intersection being a point ofintersection farthest from the start point of the trajectory amongpoints of intersection between a long axis of the ellipse and thetrajectory.
 15. The data processing apparatus according to claim 1,wherein said correction unit is configured to complement the trajectoryexpressed by the input data, to thereby determine the target region. 16.The data processing apparatus according to claim 15, wherein saidcorrection unit is configured to approximate the trajectory expressed bythe input data to an ellipse, to thereby complement the trajectory. 17.The data processing apparatus according to claim 1, wherein saidcorrection unit is configured to determine an algorithm for determiningthe target region on the basis of the input data, and correct the inputregion according to the determined algorithm, to thereby determine thetarget region.
 18. The data processing apparatus according to claim 1,wherein the data processing apparatus is configured as an integratedcircuit.
 19. A data processing method for determining a target regionwithin a picture contained in a video content, on the basis of atrajectory input by a user in order to surround a moving object in thevideo content, said method comprising: receiving input data showing thetrajectory input by the user in the video content displayed; andcorrecting an input region, which is a region specified by thetrajectory expressed by the input data, to thereby determine the targetregion.
 20. A program causing a computer to execute the data processingmethod according to claim 19, wherein the program is recorded on anon-transitory computer-readable recording medium.